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INTRODUCTIOH 



This paper introduces an outreach indicator system that can help outreach 
projects (of the U.S. Department of Education's Handicapped Children's Early 
Education Program — HCEEP) document the impact of their work. The system can 
also serve as the basis for the aggregation of impact data across outreach 
projects • 

The system does not comprise an exhaustive list of indicators. Rather, 
six indicators, gleaned from outreach project personnel over several years of 
effort, represent six primary areas of outreach activity. 

This paper briefly discusses some background information about HCEEP 
outreach projects and accomplishments. Next, the history of the "indicator" 
concept and activity is traced briefly. A description of the six impact 
indicators follows with a sample from the author's user's manual and computer 
coding system. 



THE HCEEP OUraEACH COMPONEHT 



The HCEEP outreach component began in 1972 and has received particular 
commendation for its role in stimulating the establishment of high-quality 
early intervention services for handicapped preschoolers. A report by Roy 
Littlejohn Associates (1982) summarized their evaluation of the HCEEP program 
and credited HCEEP outreach projects with being largely responsible for 
expanding early intervention services to over 100,000 preschool handicapped 
children and for generating at least 16 dollars in additional support for 
early intervention services for each dollar invested in outreach projects. 
Similarly, in an analysis of early intervention efficacy literature, the Early 
Intervention Research Institute (White, Casto, and Mastropierl, 1984) at Utah 
State University identified seven major contributions of the 22 outreach 
projects validated as exemplary projects by the U.S. Department of Education's 
Joint Dissemination Review Panel (JDRP) and approved for nationwide 
dissemination. 

To be validated by JDRP, each of the 22 outreach projects had to 
demonstrate to a panel that the interventions they developed had both 
educational and statfsti*' 1 significance. The 22 projects were originally 
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funded as HCEEP demons trat ion projects, which have no research budgets, so 
they had some difficulty collecting the necessary data. However, by using 
pre/post designs, the projects were able to collect efficacy data while 
conducting their demonstration and outreach activities. Twenty-one of these 
JDRP-approved HCEEP outreach projects are described in Table 1. The FEED 
Outreach Project was omitted because they did not work directly with 
handicapped preschoolers* 

Data summarized by White, Casto, and Mastropieri (1984) indicate that the 
JDRP-approved outreach projects have had the following national and 
internat tonal impact : 

o New services developed in every state in the nation. 

o Two thousand seventy-six projects replicated outreach models. 

o Twenty-five foreign countries replicated components of outreach models. 

o Outreach project materials have been translated into several foreign 
languages . 

o The most widely used early childhood curriculum materials were 
developed by outreach projects. 

o Hundreds of early intervention personnel have been trained. 



THE "INDICATOR" CONCEPT 

Outreach projects collect data that concerns the impact of their 
services. As a group, efforts have been focused on developing a series of 
indicators that provide a mechanism for documenting the impact of individual 
projects while allowing aggregation of the data across projects. 

In 1978, at an Outreach Project Directors* Conference at Washington, 
D.C., the first formal consideration was given to developing a comprehensive 
evaluation system for outreach projects. As a result of the conference, a 
document was produced which addressed evaluation concerns and other outreach 
parameters (Swan, 1978). The development of specific indicators of impact for 
outreach projects followed. During the next two years, these indicators were 
field tested and revised and then reviewed and refined at the 1980 Outreach 
Project Directors* Conference (Swan, 1981). 

The original indicators were based on the following five general 
principles (Swan, 1981): 

o First, indicators of impact areas should be clearly and concisely 
defined. Without such clarity, reliable and valid data cannot be 
obtained from individual projects and data cannot be aggregated 
across projects. 
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Project Name 
and Location 



Rutland Center 
Athens. Georgia 

PEECH Project 
Champaign, Illinois 

Macomb 0-3 PrOjftCt 
Macomb. Illinois 

Peona 0-3 Project 
Feoria. Illinois 

FEED Project 
Bloomington. Indiana 

ERIN Project 

Dedham. Massachusetts 

High Scope Project 
Ypsilanti. Michigan 

UNISTAPS Project 
St Paul« Minnesota 

Central institute Project 
St Louis. Missouri 

BOCES Project 
Yofktown. New York 



Preschoolers/Families Proj 
Fargo. North Dakota 

Chapel Hiit Project 
Chapel Hill. North Carolina 

Teaching Research Project 
Monmouth. Oregon 

Good Samaritan Hospit?' 
Portland. Oregon 

DEBT Project 
Lubbcik. Texas 



PEECH Project 
Wichita Falls. Texas 

MAPPS Project 
Logan. Utah 

SKrni Project 
Ogden. Utah 



Down Syndrome Project 
Seattle. Washington 

Communication Prefect 
Seattle. Washington 

C P Project 
Milwaukee. Wisconsin 

Portage Project 
Portage. Wisconsin 



Table 1 

Description of JDRP Projects 



Description 



Center-based prefect for preschoolers 
With severe emotional problems 

Center-based program for children and 
families 

Home-based program for children and 
their families 



Home-based program for children and 
their families 



Middle-grade child development curricu- 
lum for grades 7-8 

Home- & center-based early recognition 
& intervention program 

Center-based cognitively oriented pre- 
school intervention program 

Family-oriented program for hearing 
impaired 

Parent-as-teacher project for hearing 
impaired 

Comprehensive uUucational service 
program 

Four part prevention/intervention 
program 

Assessment/intervention program for 
mildly handicapped 

Individualized skills instruction program 
for moderately/severely handicapped 

Diagnostic/prescriptive classroom 
program 

Home-based/parent-taught intervention 
program 

Home-based/parent-taught intervention 
program 

Home- & center-based program for chil- 
dren tn remote areas 

Homo-based diagnostic & intervention 
program for hearing impaired 

Center-based program for Down Syn- 
drome children 

Home- & center-based program for com- 
municatively disordered children 

Center-based multidisciplinary program 
for physically disabled 

Home teaching program for multicategofi- 
cal handicapped 



Replication 



Type of Handicap Sites 

Emotionally disturbed, 2-8 78 

Mixed handicaps. 3-5 48 

Mixed handicaps, 0-3 17 

Mixed handicaps, 0-3 134 

Nonhandicapped. grades 10 
7*8 

Mixed handicaps. 2-7 60 

Mixed handicaps. 4-6 61 

Deaf & hearing impaired. 
0-5 

Hearing impaired. 0-4 32 

Mixed handicaps. 3-5 45 

Developmentally delayed* 55 
emotionally disturbed 0-6 

Mixed handicaps, 4-6 900 

Mixed handicaps, 1-18 200 

Multiply handicapped 10 

Physically handicapped 

0-6 

Mixed handicaps, 0-2 50 

Mixed handicaps, 48 
6 months-6 years 

Mixed handicaps, 0-5 60 

Hearing impaired, 0-6 98 

Down Syndrome, 0-6 52 

Communicatively 40 
disordered, 0-6 

Physically disabled. 0-3 56 

Mixed handicaps, 0-6 70 



Major Accomplishments 



Curriculum for emotionally disturbed 
Developmental therapy textbook 

Mat uals on classroom planning 
Family involvement manuals 

Baby buggy series of books & papers 
Use of mobile van to deliver services 
Rural network 

Assessment instruments 
Slide-tapes on normal & abnormal 
development 

Curriculum materials for teenagers 

Preschool screening systems 
Developmental inventory 
Developmental checklist 

Cognitively oriented curriculum 
Teacher training manual 

Home activities guide 
Preschool planning materials 

Teacher training materials 
Training institutes 

Curriculum guides 
Parent volunteer manual 
Manual & activity catalog 

Magic Kingdom Screening Program 
Parent training materials 

Learning accomplishment profile 
Mainstreaming materials 
Training materials 

Books on data-based classroom 
Teaching research curriculum 
Teaching materials 

Task analyzed curriculum mateitais 
Parent training materials 

DEBT Developmental Scale 
DEBT Teaching Activities 
Training materials 

Teachers handbook 
Parents handbook 

CAMS Curriculum Materials 
Criterion Referenced Placement Test 
Program evaluation materials 

Curriculum materials 

Identification materials 

Total communication curriculum 

Parent involvement procedures 
Teacher training materials 

Teacher training materials 
Classroom observation systems 

Prospeech assessment scale 
Training materials 

Portage Gu'do to Early Education 
Portage Parent Program 
Portage checklist 
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o Second, the smallest number of indicators possible should be used for 
each outreach activity to minimize the effort to collect and display 
the data* 

o Third, the indicators should focus on quantity. Most outreach projects 
models and components have a variety of qualitative data outputs; but 
collecting qualitative data requires significant judgment and 
interpretation which often reduces the reliability and validity of 
the information. Qualitative data should be reported project-by- 
project rather than across projects, and should be contained in 
projects' progress reports. Such qualitative data for a project 
would include assessments of model fidelity and interpretations of 
child progress data. 

o Fourth, impact indicators should focus on summative impacts rather than 
process results, because the emphasis is on documenting project efforts 
in achieving outreach goals — not on the means of achieving these 
goals. The particular combination of outreach activities used by a 
project is unique and depends on the particular outreach model, target 
audiences, model components, skills of staff, and similar variables. 

o Fifth, each indicator should be mutually exclusive from the others 

(data should be used only once), and the set should be as exhaustive as 
possible (as much should be measured as possible). While there will 
always be unanticipated results and spinoffs, the indicators must focus 
on the commonalities across projects and not project uniqueness, 
(p. 99-100) 

Many outreach projects began to collect data using the indicators, and 
they reported the data individually as part of their final reports to their 
funding agency, the U.S. Office of Special Education Programs. Unfortunately, 
the impact indicators were not specific enough to allow projects to collect 
the kinds of data that could be aggregated across all outreach projects. 
Thus, national impact of the outreach program could not be portrayed. A 
survey conducted in 1982 by the Multi-Agency Project for Preschoolers (MAPPS) 
Outreach Project at Utah State University revealed that outreach project 
directors in general were concerned about the generalizability of the impact 
indicators and were willing to help refine the system. 

In October 1983, a small group of outreach project directors met and 
outlined several steps to rewrite the outreach impact indicators and to 
develop a users' manual to accompany the indicators. This author agreed to 
rewrite the indicators relating to stimulation of high-quality programs and 
state involvment/coordination. The revised indicators were presented to 
interested outreach project directors at the December 1983 HCEEP/DEC Projects 
Conference. There, outreach project directors provided feedback on the 
revisions, and the author agreed to rewrite the other indicators in the same 
format and prepare a users' manual. 

The indicators are now in final format; data can be collected by 
interested outreach projects and forwarded to the author for analysis. The 
users' manual and a computeri2ed reporting system have also been developed. 
The users' manual, also available from the author, details specific criteria 
to be used in data collection activities. 
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DESCRIPTIOS OF IMPACT IIIPICATORS 



The present impact indicators address six major areas which "support the 
replication of established practices to assist other agencies and organiza- 
tions in expanding and improving services to handicapped children" (definition 
of outreach projects contained in final rules for HCEEP, Federal Register , 49 , 
134, July 11, 1984). The major impact areas are listed below. Descriptions 
of each area and a list of specific impact indicators follow. 

0 Promoting awareness 

o Product development and distribution 

o Stimulating high-quality programs and developing sites for 
replication 

0 Training 

o State and regional involvement and coordination 
o Other technical assistance and consultation 



Prowoting Avareness 

HCEEP outreach projects disseminate information designed to increase 
general awareness of the needs for early intervention programs for handicapped 
preschoolers and their families. The impact indicators for promoting 
awareness are: 

o Number of awareness presentations (newspaper articles, radio spots, 
television programs, speeches). 

0 Number of persons requesting additional materials or information by 
phone or letter. 

o Number of persons visiting the outreach project. 

Product DevelopBient and Distribution 

Outreach projects develop, revise, field test, and disseminate products 
ranging from awareness posters to curriculum materials. The impact indicators 
for product development and distribution are: 

o Number of print publications developed and the number disseminated. 

0 Number of audiovisual materials developed and the number disseminated. 

0 Number of children receiving new or improved services via use of 
selected project products. 
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Stittulating and Developing Sites for Replication 

A 1984 position paper, written by a group of HCEEP project directors, 
holds that site development is central to outreach efforts. Outreach projects 
help other agencies adopt components of the outreach project's model. The 
impact indicators for stimulating and developing sites for replication of the 
project's model consist of demographic and outcome data collected from the 
replication sites. 



Training 

Personnel at outreach projects train others to use selected outreach 
project model components and basic early intervention skills. The impact 
indicators for training activities are: 

o Number of people trained and intensity of training. 

o Number of state, college, or university training programs 
incorporating model components. 

o Sources and amount of funding provided by others to support training* 



State and Regional Involvement and Coordination 

Outreach projects promote state interagency coordination, networks, and 
consortia and conduct other activities designed to provide assistance to state 
departments of education and other agencies in support of state efforts in 
early education of the handicapped. Tliis impact area is assuming greater 
importance as federal initiatives increase the involvement of states in the 
planning and implementation of preschool services. The impact of indicators 
for state and regional involvement and coordination are: 

o Number of activities which facilitate improved services. 

o Number of activities which facilitate interagency cooperation. 

o Number of activities which facilitate passage of legislation. 

o Number of activities which facilitate increases in state and local 
funding. 

o Number of activities which stimulate the development of networks and 
consortia. 



Other Technical Assistance and Consultation 

Outreach projects provide technical assistance and consultation in the 
following areas: selecting child screening outcome measures, evaluation, 
sources of funding, and proposal development. These activities contribute 
indirectly to establishing new services or improving existing services. The 
impact indicators for technical assistance and consultation are: 
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0 Number and type of technical assistance consultations delivered. 

0 Number of persons or agencies who receive information on sources of 
funding or receive assistance in writing proposals and who subsequently 
receive increased funding* 



OTHER SUPPORT RESOURCES 



Users Manual 

A users' manual was developed to accompany the impact indicators and to 
ensure that comparable data could be collected across outreach projects. The 
manual explains each impact indicators. Figure 1 present? an example from the 



Figure 1 

OUTREACH IMPACT INDICATORS 
EXPLANATORY DETAIL 

This mdnudi heips ensure that compdrable data is collected across various categories by providing more detail as 
tu what information should be collected and how it should be reported. Criteria for various categories, brief explanations, 
and examples, where appropriate, are provided. 

Area 3: Stimulating Sites 

The purpose of ihis outreach aouviiy is to help other agencies and programs to improve existing services or to develop 
new services. A variety of activities are included in this category. For example, you might help an agency set up a 
partial or complete replication of your model. Or, you might be requested to offer other types of services. The table 
used to collect data on stimulating sites allows you to report any type of c^osistance you deliver. Use the following 
criteria to help you prepare table. 

Site Description 

Column 1 — For any agency or program which received assistance from you, write in the name of the agency, 
its address, and the name, address, and phone number of a contact person. 

Type of Program 

Column 2 — List the type of program according to the following description: 

1. home-based 0-3 4. combination 

2. home-based 3-5 5. other 

3. center-based 



If the program is listed under "other," attach a bnef note and try to describe the type of program. 
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manual and 
indicator 



illustrates how projects enter a portion of the data for the impact 
for stimulating and developing sites. (Outreach projects can 
contact this author for a complimentary copy of the manual; Social Integration 
Project, Exceptional Child Center^ Utah State University, UMC 68, Loean, 
Utah 84322. y b j 



Computerized Data Collection 

A form to accompany the users' manual was developed so that data could be 
stored and analyzed by computer. This computer data sheet simplifies the data 
collection process and provides a permanent data base for outreach projects. 
The system was designed so that as outreach objectives change, new impact 
indicators may be added or substituted for the present ones. (This form is 
also available from the author.) 



SUMHARY 



The HCEEP outreach impact indicators represent an attempt by outreach 
projects to demonstrate accountability and provide information that documents 
their impact on the field of early intervention. Though the indicators do not 
represent the entire spectrum of outreach activities, they do allow for the 
collection and reporting of certain data across outreach projects. This 
information provides a long overdue data base for future planning. 
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